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Background / Context: 

The problem of summer reading loss among low-ineome children has been amply documented 
by researchers. On average, summer vacation creates a 3 -month gap in reading achievement 
scores between low-income and middle-income children (Cooper, Nye, Charlton, Lindsay, & 
Greathouse, 1996). Over time, income-based disparities in reading achievement appear to grow 
more rapidly during the summer months than the academic school year (Alexander, Entwisle, & 
Olson, 2007; Downey, von Hippel, & Broh, 2004). To accelerate the reading gains of low- 
income children, researchers have recently designed and evaluated interventions that encourage 
children to read books at home during summer vacations (Wilkins et ah, 2012). Efforts to 
promote children’s book reading activities at home are based on sound experimental evidence 
(Allington et ah, 2010). Numerous correlational studies indicate that regular exposure to print 
over the life span is a strong predictor of reading comprehension and verbal ability (Eindsay, 
2010; Mol & Bus, 2011; Stanovich, 2000). Recent experimental evidence, however, has yielded 
conflicting flndings on the potential efficacy of teacher-scaffolding of summer book reading. A 
recent longitudinal experiment indicated that children who received 12 self-selected books over 3 
consecutive summers enjoyed significant reading comprehension gains, suggesting that teacher- 
scaffolding lesson may be unnecessary to improve comprehension (Allington et ah, 2010). 
However, a large statewide involving 1,785 low-income 3'^‘* graders in Texas (Wilkins et ah, 
2012) showed no impact on comprehension when children received only matched books. 

We conducted two randomized experiments that were designed to go beyond the question — does 
access to books improve comprehension among elementary school children? Instead, we 
conducted two experiments in which teachers scaffold summer book reading with two different 
approaches to improving comprehension through end of school year lessons. Because teachers 
implemented different comprehension lessons in the two studies, the instructional activities 
varied across the two studies. In addition, both studies used the same norm-referenced reading 
comprehension test (Iowa Test of Basic Skills), enabling us to compare impacts on a similar 
measure across two studies. 

Purpose / Objective / Research Question / Focus of Study: 

The purpose of this study is to capitalize on the lessons from two experiments. Results from our 
2006 experiment, which focused on comprehension strategy instruction, improved children’s 
performance on comprehension tasks that required inference and interpretation {d = .15). Results 
from our 2011 experiment suggest that content-oriented comprehension instruction improved 
children’s performance on tasks that required analysis and generalization {d= AV) compared to 
strategy-oriented instruction. In our presentation, we will review the results of the first study and 
focus our attention on our more recent experiment, which compares the content-oriented lessons 
to the strategy-oriented lessons. 

Setting: 

Study 1 was conducted in a large, multi-ethnic school district in metropolitan Washington, DC 
and involved 552 children in 10 K-6 elementary schools. Study 2 was conducted in a mid-sized 
urban district in North Carolina and involved 981 Grade 3 children in 19 K-5 elementary 
schools. 
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Population / Participants / Subjects: 

The study 1 and study 2 descriptive characteristics are displayed in table 1 and table 2, 
respectively. Comparison of the two study samples indicates that children in the North Carolina 
were more likely to be low-income than children in the metropolitan Washington, DC district. 
The reading scores were near the national norm in both samples. 

Intervention / Program / Practice: 

In study 1 , Grade 4 teachers implemented strategy-oriented comprehension lessons and children 
received books matched to their reading level and interests during the summer months. During 
the training, the lead teacher used the children’s book. The Wreck of the Zephyr, to model five 
comprehension strategies: re-reading, asking questions, making predictions, summarizing, and 
making connections to self and to other text. Second, teachers were asked to instruct their 
students about paired reading (National Reading Panel, 2010), a widely used oral fluency 
strategy in which a student chooses a favorite part of a book (100 words) to read out loud to a 
parent or family member. Teacher lessons occurred during the last month of school. 

In study 2, we compared the strategy-oriented lessons with content-oriented lessons. Teachers in 
both lesson conditions used the same narrative text {Look Out, Jeremy Bean) and the same 
informational texts {Polar Bears] The Sweaty Book of Sweat). Teachers for both the content and 
strategy groups attended a 2-hour training session, during which they received 6 lesson plans and 
materials. Thus, in study 2, the strategy lessons were modified and included instruction with two 
genres of text (narrative and informational). 

The strategy lessons in study 2 largely replicate the lessons in study 1 and were also applied to 
informational texts. In the strategy condition, training was led by the two teachers who led 
training in study 1. Teachers learned how to implement three lessons involving comprehension 
strategies for reading the narrative text and the two informational texts, and instructed children to 
use a simple postcard to answer questions about their books. For the postcard activity involving 
fiction books, teachers read aloud from the lessons books and instructed children to write down 
the book title, and indicate whether they finished the book, how many times they read the book, 
and whether they used comprehension strategies to better understand the book, including re- 
reading, making predictions, asking questions, and making connections (text to text, text to self). 
Next, children were instructed to tell someone in their family what the book was about. Then 
they were instructed to select a 100-word excerpt from the book and to read aloud to a family 
member. 

In the content condition, training was led by two veteran teachers in the North Carolina district, 
including a National Board Certified teacher and an instructional coach. Trainers instructed 
teachers to apply two comprehension routines with texts used in classroom lessons and with texts 
read at home during the summer. The content-oriented lessons had a stronger emphasis on 
helping children to integrate ideas in texts and to make generalizations. Teacher trainers 
explained that the lessons were designed to draw students’ attention to structural differences 
between narrative and expository texts, use different tools to identify structural features unique to 
each type of text, and motivate engagement with text during reading activities. For story 
impressions, the accuracy of the guess was not as critical as encouraging engagement with text 
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and deeper comprehension (Anderson & Pearson, 1984). A critical difference between the 
narrative and expository routine in the content lessons was the amount of book-specific 
scaffolding that a child received. Because story impression words are specific to the text that a 
child is reading, each narrative book included a postcard and a book-specific story impression. 
The second author trained research assistants who generated 356 story impressions for each of 
the narrative texts. For the expository texts, children were prompted to use the KWL chart to 
activate background knowledge and to integrate it with text-based information. 

During the last two weeks of school, children participated in 6 comprehension lessons that 
focused either on content or strategy approaches to comprehension. We assessed the percentage 
of lesson components that were adhered to by teachers implementing content- and strategy- 
oriented comprehension lessons. We fit a multi-level model with school-level random effects to 
compare adherence scores. Our results indicate that the mean adherence rate among teachers 
implementing content (74%) and strategy lessons (70%) was statistically equivalent (Coefficient 
= .037, SE = .05, p = .457). 


Research Design: 

Study 1 was a student-level randomized experiment. A total of 552 children were randomly 
assigned to a treatment group children, who received 8 matched books and postcards in the mail 
during the summer months. 

Study 2 was a cluster-randomized experiment that was part of a larger study on summer reading. 
In particular, study 2 is part of a larger randomized experiment that is designed to examine (a) 
the effects of two different comprehension approaches and (b) the effects of additional 
scaffolding of summer book reading involving follow-up teacher phone calls. In spring 2011, 19 
K-5 elementary schools were stratified by poverty and achievement levels and randomly 
assigned to implement either content- or strategy-oriented comprehension lessons. After schools 
were assigned to lesson condition, children and teachers were randomly assigned to (a) an 
untreated control condition, (b) a classroom lesson and summer book condition, (c) or a 
classroom lesson, summer book, and teacher call condition. A second study will focus on the 
effects of the third condition involving teacher phone calls. 

In the results reported for study 2, we focus on the first goal of comparing school-level 
comparisons of the content- and strategy-oriented lessons. We pooled data from the two 
experimental conditions , including (b) a classroom lesson and summer book condition and (c) a 
classroom lesson, summer books, and teacher calls conditions, because both groups of children 
received the lessons and summer books. In condition (c), because only 1 1% of the children 
received 3 teacher phone calls, there was little difference in the intervention activities that were 
part of the two experimental conditions. More importantly, there was no interaction between 
classroom lesson condition and student phone call condition on comprehension outcomes. 
Therefore, in subsequent analyses, we combined data for children in the two experimental 
conditions. The school, teacher, and student sample size were the following; In the 9 content- 
oriented lesson schools, 461 children were nested within 24 classrooms, yielding an average 
class size of 19 children. In the 10 strategy schools, 520 children were nested within 30 
classrooms, yielding an average class size of 17.33 children. Children who remained in the study 
at posttest were administered reading tests and surveys in the second week of September 2011 at 
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the beginning of fourth-grade on the Iowa Test of Basic Skills (ITBS), a standardized 
comprehension test. 

Data Collection and Analysis: 

In study 1 and study 2, children completed pretests during the last 2 weeks of the school year and 
posttests in the second week of the school year. The Iowa Test of Basic Skills was administered 
to consented students. We disaggregated the comprehension tests into three subtests: factual 
understanding items, inference and interpretation, and analysis and generalization. Cronbach’s 
alpha reliabilities were above .80 for each subtest. 

Findings / Results: 

We used OLS regression with classroom randomization blocks and a pretest covariate to analyse 
data from study 1 . The results from study 1 indicate that children in the treatment group, which 
received strategy-oriented comprehension lessons and 8 books during the summer, enjoyed a d = 

. 1 5 gain in comprehension tasks related to inference and interpretation. There were no gains in 
factual understanding or analysis and generalization. 

The results from study were based on a school-level random effects model including fixed- 
effects for school poverty, pretest reading scores, and lesson condition, and random-effects to 
account for the clustering of children within classrooms and classrooms within schools. We fit 
the following mixed effect model 

Yijk = Yoo + Yoi (Pretest ITBS)i+ Yo 2 (School Poverty)* + Yo 3 (Condition)*+ (|Uk + 5jk + Sijk), (1) 

where Y^k represents the posttest score of student i in classroom j in school k, and Pretest ITBS 
and School Poverty represent covariates that were included to improve the precision of the 
estimated treatment effect on the coefficient yo 3 , which captures the estimated difference in 
posttest scores between students participating in content or strategy lessons. The baseline 
covariates and the dummy variable for condition were modeled as fixed effects and the school- 
(pk), classroom- (5jk) and student-specific error terms (Sijk) were modeled as random effects. 
Using the mixed effects model (1), we estimated short-term effects measured in the fall of 
fourth-grade on total comprehension and narrative and expository comprehension. The impact 
estimates on each of the three posttest outcomes. There was a significant positive impact on 
analysis subtests {d = .12, SE = .06), and no impact on factual understanding {d = .03, SE = .05) 
or inference and interpretation {d = .05, SE = .06). 

Conclusions: 

Although previous experimental studies on voluntary summer reading have yielded mixed 
findings (Allington et ah, 2010; Wilkins et ah, 2012), the results of two experimental studies of 
teacher-scaffolded summer reading show how comprehension instruction affects reading 
comprehension among elementary school children. In study 1, strategy lessons appeared to 
benefit Grade 4 children’s ability to make inferences and interpretation. In study 2, content 
lessons appeared to benefit Grade 3 children’s ability to make analyses and generalizations. We 
capitalized on exogenous variation in the quality of teacher instruction right before summer 
vacation to reveal how summer book reading improves different aspects of children’s reading 
comprehension. 
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Table 1: Study 1 Demographic Characteristics of Children (N = 552) 


Variable 

% M SD 

Female 

47 

White 

33 

Blaek 

19 

Latino 

26 

Asian 

17 

Other 

05 

Free-Redueed Luneh 

39 

Limited English Profieiency 

38 

Iowa Test of Basie Skills (NPR) 

52 28 

Note. NPR = National Pereentile Rank 

Table 2: Study 2 Demographic Characteristics of Children (N = 981) 

Characteristie 

% M SD 

Female 

48 

Free-Redueed Luneh 

73 

White 

15 

Black 

49 

Latino 

32 

Asian 

2 

Multiracial 

2 

Limited proficient with English 

30 

Reading comprehension (NPR) 

45 29 

Note. NPR = National Percentile Rank 
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